Categorical and specificity differences between user-supplied tags and search query terms for images. An analysis of Flickr tags and Web image search queries

A) Abstract object (A1) Mythical or fictitious being (A1) Dragon Emotion/Abstraction (A2) Symbolic value (A2-1) Classic Page 6 of 22 Categorical and specificity differences between user-supplied tags and search query terms... 12/23/2009 http://informationr.net/ir/14-3/paper408.html For comparing the level of term specificity, the basic level theory was adopted. The basic theory explains that concepts can be categorized into one of three levels, the superordinate level, the basic level or the subordinate level. Experimental studies have demonstrated most people tend to use the basic level concept rather than the superordinate or subordinate concept (Rosch et al. 1976). Since it has been found that a set of basic level terms are dominantly used and commonly shared by general users, researchers in library and information science assumed that basic level terms should be the level of specificity for concepts, and should receive focus during the indexing process (Bates 1998; Green 2006). By following this assumption, this study examined the level of specificity of user-supplied tags and search terms by applying the basic level theory. General feeling, atmosphere (A22) Cold Individual affection, emotional cue (A2-3) Happy Abstract location (A3) Place symbolized (A3) Urbanlocation (A3) Place symbolized (A3) Urban Abstract time (A4) Emotion, abstraction symbolized by time (A4) -time (A4) Emotion, abstraction symbolized by time (A4) Generic (G) Generic object (G1) Kind of person, people, parts of a person (G1-1) Baby Kind of animal, parts of an animal (G1-2) Bear Kind of thing (G1-3) Airplane Generic event/activity (G2) Kind of event (G2-1) Birthday Kind of action (G2-2) Bowling Generic location (G3) Kind of place (G3) Beach Generic time (G4) Cyclical time, time of day (G4) Morning Specific (S) Specific object (S1) Individually named person (S1-1) Chris Individually named animal (S1-2) Heron Individually named thing (S1-3) Sega Specific event/activity (S2) Individually named event (S2-1) Olympic Individually named action (S2-2) Specific location (S3) Individually named geographic location (S3) Florida Specific time (S4) Linear time (date or period) (S4) 2007 [Others] Colour (C) Black Boolean + search command (B) AND, Find Image related (I) Photo etc. Flickr related (F) Geotag Number (N) 1 Part of speech (P) And Table 3: Category of pictorial meaning Page 7 of 22 Categorical and specificity differences between user-supplied tags and search query terms... 12/23/2009 http://informationr.net/ir/14-3/paper408.html Since most research on basic level theory has explored concrete objects and colours, this study also analysed tags and search terms in the Generic and Colour categories. Rosch and her colleagues demonstrated features of superordinate, basic and subordinate categories through their empirical studies (Rosch et al. 1976), but they did not provide established criteria which can clearly distinguish those three categories; whereas, in information scince, some recent studies developed their coding schemes for applying the basic level theory (Green 2006; Rorissa 2008; Rorissa and Iyer 2008). This study attempted to establish a coding scheme which reflects an existing hierarchical structure among concepts in addition to considering features of three categories illustrated by previous studies. This study made use of the hierarchies appearing in the Library of Congress Thesaurus for Graphic Materials (hereafter, 'the Thesaurus') by following three steps. First, it examined how nine taxonomies used in the empirical study of Rosch et al. (1976) are designated in the the Thesaurus hierarchy (Figure 1). We found that the absolute level of depth appearing in the Thesaurus hierarchy cannot be directly used in deciding three categories. For example, in the case of an Object → Food → Fruit → Apple hierarchy, the lowestlevel word, Apple, obviously satisfies features of the basic level, the upper three concepts belong to the superordinate level and this hierarchy does not include a subordinate level. The examples of Hammer, Saws and Crosscut saws shows that two basic-level terms, Hammer and Saws, belong to two different levels in the Thesaurus and a subordinate term Crosscut saws is placed at the same level as Hammer. Although the absolute depth of the Thesaurus's hierarchy cannot be a criterion for deciding basic-level categories, it was obvious that considering its hierarchical relations among concepts can help make decisions on basic levels. Therefore, secondly, other tags and terms not included in nine taxonomies but found in the Thesaurus were categorized into one of three levels. This was done by considering features of the three categories as well as Thesaurus hierarchies. Finally, tags and terms not included in the Thesaurus were also categorized in a consistent way (refer to Yoon(2009) for a more detailed explanation). With regards to basic level colours, the analysis process was more straightforward because eleven basic colours were identified in a previous study (Berlin and Kay 1969): black, white, grey, red, yellow, green, blue, pink, orange, brown, and purple. Page 8 of 22 Categorical and specificity differences between user-supplied tags and search query terms... 12/23/2009 http://informationr.net/ir/14-3/paper408.html Figure 1. TGM hierarchy and Rosch's nine taxonomies. Bolded concepts exist in nine taxonomies used by Rosch et al. (1976) Categories and term specificity were coded by a trained masters' level student in the School of Library and Information Science at the University of South Florida. For checking the reliability of the coding for categorical analysis, two methods were used. First, tags and query terms were sorted by attribute and then alphabetical order, and then one of the researchers reviewed the coding, discussed with the student the anomalous codes and corrected anomalous codes (error rate < .01%). Secondly, another trained masters' level student in the same school performed coding checks on 10% of the records. The percentages of inter-coder agreement were 92% for user-supplied tags and 96.4% for search terms. The reliability of the coding for term specificity analysis was checked by examining inter-coder agreement. Again a trained masters' level student in the same school performed coding checks on 10% of the records. The percentages of inter-coder agreement were 89% for both user-supplied tags and search terms. Page 9 of 22 Categorical and specificity differences between user-supplied tags and search query terms... 12/23/2009 http://informationr.net/ir/14-3/paper408.html